Skip to content

Use upstream Rocky Linux base images for rocky driver containers - #880

Open
abrarshivani wants to merge 1 commit into
NVIDIA:mainfrom
abrarshivani:rocky-upstream-base-image
Open

Use upstream Rocky Linux base images for rocky driver containers#880
abrarshivani wants to merge 1 commit into
NVIDIA:mainfrom
abrarshivani:rocky-upstream-base-image

Conversation

@abrarshivani

@abrarshivani abrarshivani commented Jul 24, 2026

Copy link
Copy Markdown

What type of PR is this?

/kind cleanup

What this PR does / why we need it:

The rocky8/rocky9/rocky10 driver images are built on nvcr.io/nvidia/cuda:*-base-rockylinux* — a CUDA runtime image used only as "a Rocky image on a registry we control". This points them at Rocky Linux's own images instead, matching how the rhel targets build on Red Hat UBI.

build-rocky8%:  BASE_IMAGE=rockylinux/rockylinux:8.10-ubi
build-rocky9%:  BASE_IMAGE=rockylinux/rockylinux:9.8-ubi
build-rocky10%: BASE_IMAGE=rockylinux/rockylinux:10.2-ubi

The current setup ties Rocky driver builds to the CUDA image release cadence, and that is already causing drift: 13.2.1-base-rockylinux10 and 13.3.0-base-rockylinux10 were never published, which is why rocky10 is pinned to 13.2.0 while rocky8/rocky9 are on 13.3.0. If the rockylinux CUDA variants are ever dropped, Rocky builds break outright.

This also adds Renovate guards. All three Rocky majors share a single image repository, so an unconstrained bump could move rocky9 onto a Rocky 10 tag — the old nvcr.io tags were implicitly protected by docker versioning's suffix compatibility.

Which issue(s) this PR fixes:

Fixes #764

Special notes for your reviewer:

Why the -ubi flavor. The rhel Dockerfiles are written against Red Hat UBI, and Rocky's -ubi flavor is built to match the UBI package manifest, so the shared Dockerfile works unmodified. The default Rocky flavor lacks which (used by ocp_dtk_entrypoint) and systemd, both of which UBI and -ubi ship.

Nothing the driver container used is lost. Diffing the RPM databases, cuda:13.3.0-base-rockylinux9 is exactly rockylinux/rockylinux:9 plus cuda-cudart, cuda-compat, three cuda-toolkit-*-config-common, libxcrypt-compat and a gpg-pubkey. None are referenced anywhere under rhel{8,9,10}/ — those Dockerfiles do not even declare ARG CUDA_VERSION, so the --build-arg CUDA_VERSION the Makefile passes is already discarded as unused.

The CUDA package repo is untouched. install.sh setup_cuda_repo adds it from a hardcoded developer.download.nvidia.com URL at build time, independent of the base image — which is why the rhel builds already work on plain UBI with no CUDA base anywhere. The base image's own cuda.repo was already deleted before the final layer. install.sh also already handles ID=rocky → enable crb, and dnf config-manager arrives as a weak dep of epel-release in both the old and new base. The precompiled paths genuinely need CUDA images and are not touched.

What I verified. make -n resolves all three rocky targets to the new bases, and rhel8/rhel9/rhel10 still pass no override and keep their UBI defaults. All three pinned tags exist on Docker Hub with amd64 and arm64. renovate.json is well-formed.

What I could not verify — please /ok to test. Draft because the images have not been built; no container runtime was available to me. The image workflow building rocky8/9/10 across both driver versions and arches is the real gate.

On registries. This uses the public rockylinux/rockylinux reference rather than an internal mirror. dockerhub.nvidia.com resolves only on the NVIDIA network, so hardcoding it would break external contributors, customers rebuilding driver images from source, and Renovate. Docker Hub is already a build dependency here via the ubuntu:* bases (and docker:dind in GitLab CI), so this adds no new registry. If internal pull-through caching is wanted it belongs in the runners' buildkitd.toml as a docker.io mirror, where it would transparently cover the Ubuntu pulls too.

Does this PR introduce a user-facing change?

The rocky8, rocky9 and rocky10 driver images are now built on upstream Rocky Linux base images (rockylinux/rockylinux:<version>-ubi) instead of CUDA base images. The driver, Fabric Manager, NSCQ, nvsdm and IMEX packages are unchanged.

The rocky8/9/10 driver images were built on nvcr.io/nvidia/cuda:*-base-
rockylinux*, which coupled Rocky driver builds to the CUDA image release
cadence. That coupling is already biting: no 13.2.1 or 13.3.0
base-rockylinux10 tag was ever published, which is why rocky10 was pinned
to 13.2.0 while rocky8/9 moved to 13.3.0.

Point the rocky targets at Rocky Linux's own images instead, mirroring how
the rhel targets build on registry.access.redhat.com/ubi*. The -ubi flavor
is used because its package set matches the Red Hat UBI images that the
rhel Dockerfiles are written against - notably it ships `which` and
systemd, which the default Rocky image does not.

This drops nothing the driver container used. The CUDA base only added
cuda-cudart, cuda-compat and the cuda-toolkit-*-config-common packages on
top of the plain Rocky image; none are referenced by the rhel Dockerfiles,
which do not even declare ARG CUDA_VERSION. The CUDA package repository
that supplies fabricmanager, nscq, nvsdm and imex is unaffected: it is
added explicitly by install.sh setup_cuda_repo at build time and is
independent of the base image. The base image's own cuda.repo was already
being deleted before the final layer.

Renovate needs new guards because all three Rocky majors share a single
image repository, so an unconstrained bump could move rocky9 onto a Rocky
10 tag. The previous nvcr.io tags were implicitly protected by docker
versioning's suffix compatibility.

Signed-off-by: Abrar Shivani <ashivani@nvidia.com>
@abrarshivani
abrarshivani marked this pull request as ready for review July 24, 2026 23:19
@abrarshivani abrarshivani self-assigned this Jul 24, 2026
@abrarshivani
abrarshivani requested a review from rahulait July 24, 2026 23:19
@tariq1890

Copy link
Copy Markdown
Contributor

Thanks @abrarshivani! The changes look good to me. Can we paste the logs of running driver containers on RockyLinux systems built from these changes?

Once done, these should be good to merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Enhancement]: move away from cuda based images for rocky and use upstream UBI images

2 participants